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Abstract. In this paper, we study the Plancherel measures of a class of non-connected nilpotent 
groups which is of special interest in Gabor theory. Let G be a time-frequency group. More precisely, 
that is G = (Ttc,Mi : k £ Z'',/ £ BZ"*^ , Tk, Mi are translations and modulations operators acting 
in L^(R'^), and B is a non-singular matrix. We compute the Plancherel measure of the left regular 
representation of G which is denoted by L. The action of G on L^(R'') induces a representation 
which we call a Gabor representation. Motivated by the admissibility of this representation, we 
compute the decomposition of L into direct integral of irreducible representations by providing 
a precise description of the unitary dual and its Plancherel measure. As a result, we generalize 
Hartmut Fuhr's results which are only obtained for the restricted case where d = 1, B = 1/L, L £ Z 
and L > 1. Even in the case where G is not type I, we are able to obtain a decomposition of the left 
regular representation of G into a direct integral decomposition of irreducible representations when 
d = 1. Some Interesting applications to Gabor theory are given as well. For example, when B is an 
integral matrix, we are able to obtain a direct integral decomposition of the Gabor representation 
of G. 



1. Introduction 

Let G be a locally compact group with type / left regular representation. The Plancherel theorem 
guarantees the existence of a measure fj, on the unitary dual of G such that once a Haar measure is 
fixed on the group G, fj, is uniquely determined. Although the existence of the Plancherel measure 
is given; it is generally a hard but interesting problem to compute it. Let G be the unitary dual of 
the group G. The group Fourier transform f ^ f maps {G) n {G) into Jg %■„: ® HTrdfj, (vr) . 
Also, the group Fourier transform extends to a unitary map defined on (G). This extension is 
known as the Plancherel transform V. For arbitrary functions / and g & (G) , the following 
holds true: 



if, 9) 



Moreover, if L is the left regular representation of G acting in (G) then we obtain a unique 
direct integral decomposition of L into unitary irreducible representations of G such that 



Vo LoV'^ = vr (g) In^ dfj, (vr) . 
Jg 



Let B be an invertible matrix of order d. For A: G Z'', and I £ Bli'^, we define the following unitary 
operators T^, Mi : (R'^) ^ (R'^) such that T^/ (x) = f (x - k) and Mif (x) = e^^^^''^')/ (x) . 
Let G be the group generated by T^, and M/. T^. is a time-shift operator, and Mi is a modulation 
operator (or frequency shift operator). We write 



G = (Tk,Mi \ k gZ^J £ BZ'^j . (1) 

Characteristics of the closure of orbits of the type G{f) where / G L'^{W^) have been studied in 
[2] when B only has rational entries. Also a thorough presentation of the theory of time-frequency 
analysis is given in [9]. In this paper, we are mainly interested in the following questions. 
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Question 1.1. If G is type I, can we provide a description of the unitary dual of G, and a precise 
formula for the Plancherel measure of G? 

Question 1.2. If G is not type I, can we obtain a decomposition of the left regular representation 
into unitary irreducible representations of G? 

For obvious reasons, we term the group G a time-frequency group. There are three cases 
to consider. First, it is easy to see that G is a commutative discrete group if and only if B is an 
integral matrix. In that case, all irreducible representations of G are characters, and thanks to 
the Pontrjagin duality, the Plancherel measure is well-understood. In fact if S is a matrix with 
integral entries, the Plancherel measure of G is supported on a measurable fundamental domain of 
the lattice Z'^ x (S-i)*''Z'^; namely the set 

Interestingly, it can be shown that the Gabor representation 

Z"^ X BZ'^ B {k, l)^TkO Ml 

is unitarily equivalent with a subrepresentation of the left regular representation if and only if 
|det-B| < 1. Otherwise, the Gabor representation is equivalent to a direct sum of regular repre- 
sentations. Although the previous statement is not technically new, the proof given here is based 
on the representation theory of the time- frequency group. Secondly, if B has some rational non 
integral entries, then G is a non-commutative discrete type I group. Using well-known techniques 
developed by Mackey, and later on by Kleppner and Lipsman in [T2], precise descriptions of the 
unitary dual of G and its Plancherel measure are obtained. Two main ingredients are required to 
compute the Plancherel measure of G. Namely, a closed normal subgroup of N whose left regular 
representation is type I, and a family of subgroups of G/N known as the 'little groups'. We will 
show that the Plancherel measure in the case where B has some non integral rational entries but 
no irrational entry, is a fiber measure supported on a fiber space with base space: the unitary dual 
of the commutator subgroup of G. That is, the base space is [G, G]. Using some procedure given 
in [8j, we can construct a non-singular matrix A, such that AZ"^ = Z'^ n {B^^^ and we show 
that each fiber can be identified with some compact set 

Ai X X {xa} C M'' X M'^ X {xa} 

where Xa ^ [G^jCli is a measurable fundamental domain of {^A~^Y^ Z^ and Eg- is the cross- 
section of the action of a little group (which is a finite group here) in j (i?"^)**^ Z'^ . We also show 
that all irreducible representations of G are monomial representations modelled as acting in some 
finite-dimensional Hilbert spaces with dimensions bounded above by |det A|. It is worth noticing 
that Hartmut Fiihr has already computed the Plancherel measure of the simplest example (Section 
5.5 of [6j) of the class of groups considered in our paper. In his example, d = \ and BZ^ = 1/LZ 
where L is some positive integer greater than one. For the more general case in which we are 
interested, we obtain a parametrization of the unitary dual of G, and we derive a precise formula 
for the Plancherel measure. In the case where G is not type I {B has some irrational entries); 
unfortunately the 'Mackey machine' fails. In the particular case where d = 1, 

G = (Tfc, Mi I A: G Z, / e aZ) , a G M - Q, 

we are able to obtain a direct integral decomposition of the left regular representation of 
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In fact, we derive that the left regular representation of G is unitarily equivalent to 

/ / lnd'K{x(\m)\Adtd\ (2) 



[-1,1] J[0,|A|) 

acting in J®-,^ ^-^ -^[0 |A|) (^) 1"^! dtdX where 

r r 1 / 

K =1 1 

I L 1 

and 

"10/ 



X{\x\,t) 



: {k,l) G 



exp {2TTi{\X\l + tk)) . 



1 k 
1 

To the best of our knowledge, the decomposition given in ([2]) is appearing for the first time in the 
literature. 
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2. Preliminaries 

Let us start by setting up some notation. Given a matrix A of order d, A^^ stands for the 
transpose of A, and jd"*** = (^A~^Y^ stands for the inverse transpose of A. Let F be a field or a 
ring. It is standard to use GL (d, F) to denote the group of invertible matrices of order d with 
entries in F. Let G be a locally compact group. The unitary dual, which is the set of all irreducible 
unitary representations of G is denoted by G. Given x E W^, we define a character which is a 
one-dimensional unitary representation of into the one-dimensional torus as Xx '■'^'^ ^ where 

(y) = e2'^^<^'?'>. Let H he a subgroup of G. The index of F in G is denoted by {G : H) = \G/H\ . 
We will use 1 to denote the identity operator acting in some Hilbert space. Given two isomorphic 
groups G, H we write G = H. 

The reader who is not familiar with the theory of direct integrals is invited to refer to [5]. 

Definition 2.1. Given a countable sequence {fi}^^j of functions in a Hilbert space Ti, we say 
{/ilie/ forms a frame if and only if there exist strictly positive real numbers A,B such that for 
any function / G ^ 

iG/ 

In the case where A = B, the sequence of functions is called a tight frame, and if A = 

B = 1, {/jjjgj- is called a Parseval frame. 

Remark 2.2. If {fi}i^j is a Parseval frame such that for all i G I, \\fi\\ = 1 then {/i}ie/ ^''^ 
orthonormal basis for % . 

Definition 2.3. A lattice A in is a discrete additive subgroup of . In other words, every 
point in A is isolated and A = AIJ^ for some matrix A. We say A is a full rank lattice if A is 
nonsingular, and we denote the dual of A by A~^^'^A. A fundamental domain (or transveral) 
D for a lattice in is a measurable set such that the following hold: 

{D + l)r\{D + I') / 

for distinct /, /' in A, and = IJ {D + 1) . We say D is a packing set for A if xd {x — I) <! 
for almost every x or equivalently if (D + l)n{D + I') has Lebesgue measure zero for any I ^ I' . Let 
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m be the Lebesgue measure on M'^, and consider a full rank lattice A = A'Z'^ inside W^. The volume 
of A is defined as vol (A) = m (M°'/A) = |det A| . The density of A is defined as d{A) = \det A\^^ . 

Definition 2.4. Let A = AH^ x BU^ be a full rank lattice in M^d and g e L"^ (M'*) . The family of 
functions in Lp' 



is called a Gabor system. 

Definition 2.5. Let m be the Lebesgue measure on M'^, and consider a full rank lattice A = AZ'^ 
inside M*^. 



(a) The volume of A is defined as vol (A) = m (M /A) = |det j4| . 

(b) The density of A is defined as d{A) = ^ ^| • 

Lemma 2.6. Given a separable full rank lattice A = AU^ x BZ"^ in M^''. The foUowings are 
equivalent 

(a) There exits / G L^^j^d) ^^^^ ^Yiat G {f,AZ'^ x BZ'^) is a Parseval frame in 

(b) vol {A) = IdetAdet^l < 1. 

(c) There exists / G {W^) such that G (/, AZ'^ x BZ'^) is complete in {W^) . 



Proof. See theorem 3.3 in |10] . □ 

Lemma 2.7. Let A be a full-rank lattice in M^. There exists a function f £ L"^ (M"^) such that 
G if, A) is an orthonormal basis if and only if vol (A) = 1. Also, if G (/, A) is a Parseval frame for 
L^{R'^) then \\f\\^ = vol{A). 

Proof See □ 
Let 

F = (T^,My I X G M'^,?/ G IE 

and let / be a square-integrable function over M'^. It is easy to see that 

T^MyT-Hly^f = e-^^'^y^^^f, (4) 

and e"^'^*^^'^^ is a central element of the group F. Thus F is a non-commutative, connected, non- 
simply connected two-step nilpotent Lie group. In fact F is isomorphic to the reduced 2d + 1 
dimensional Heisenberg group. The 2d + 1-dimensional Heisenberg group has for Lie algebra 

\) = M-span {Xi, • • • , XdY^, ■■■ ,Yd,Z} 

with non-trivial Lie brackets [Xj, Yj] = Z. Let 

M = exp ([)) (5) 

Clearly, H is a simply connected, connected 2n + 1-dimensional Heisenberg group, and exp (ZZ) is 
a discrete central subgroup of M. Moreover, H is the universal covering group of F. That is, F is 
isomorphic to the group H/ exp (ZZ) . 

We will now provide a light introduction to the notion of admissibility of unitary representations. 
A more thorough exposure to the theory is given in [6]. Let vr be a unitary representation of a 
locally compact group X, acting in some Hilbert space H. However, we will discuss part of the 
material which is necessary to fully understand the results obtained in our work. We say that vr is 
admissible, if and only if there exists some function (p £ H such that the operator IV^ 

W^:H^L^ {X) and W^ip (x) = vr (x) (/>) . 
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defines an isometry on H. Let (p, /C) be an arbitrary type I representation of tlie group 

G = (Tk,Mi :k£Z'^,l£ BI'^^ 



Moreover, let us suppose that X is a type I unimodular group. Let us also suppose that we are 
able to obtain a direct integral decomposition of p as follows 



Jx 



According to well-known theorems developed in [6]; (/?, /C) is admissible if and only if it is unitarily 
equivalent with a subrepresentation of the left regular representation, and the multiplicity function 
is integrable over the spectrum p. That is: Ji is absolutely continuous with respect to the Plancherel 
measure p supported on X and riT^dJI {n) < oo. If a representation p is admissible, in theory 
it is known (see [6]) how to construct all admissible vectors. Let us describe such process in 
general terms. First, we must construct a unitary operator 



Jx 



intertwining the representation p with n-^T^ djl{7r) . Next, we define a measurable field {F;s^}^gjj 

of operators in (B^^iT-L^ dp (^) such that each operator Fx is an isometry. All admissible vectors 

are of the type U^^ {{^x}xex) ■ 

In the remainder of the paper, we will focus on time-frequency groups. We will compute the 
Plancherel measure of the group whenever G is type I, and we will obtain a direct integral decom- 
position of the left regular representation if G is not type I and d = 1. Some application to Gabor 
theory will be discussed throughout the paper as well. 

3. Normal Subgroups of G 

In this section, we will study the structure of normal subgroups of the time frequency group. 
The reason why this section is important is because part of what the Mackey machine [13j needs 
is an explicity description of the unitary dual of type I normal subgroups in order to compute the 
unitary dual of the whole group. In this paper, unless we state otherwise, G stands for the following 
group: 

(Tk,Mi \ k£Z'^,l£ BX^^ . 

We recall that the subgroup generated by operators of the type o M/ o T^-*^ o Mj^"^ is called the 
commutator subgroup of G and is denoted [G, G] . From now on, to simplify the notation, we will 
simply omit the symbol o whenever we are composing operators. 

Lemma 3.1. [G,G] is isomorphic to a subgroup of the torus. 

(a) If B £ GL (d, Z) then G is commutative and isomorphic to TL'^ x BTL'^ . 

(b) If B is in GL (d, Q) — GL (d, Z) then [G, G] is a central subgroup of G, and is a cyclic group. 
G is not commutative but it is a type I discrete unimodular group. 

(c) If B £ GL (d, M) — GL (d, Q) then, G is a non-commutative two-step nilpotent group, and 
its commutator subgroup is a non finite subgroup of the circle group. 

Proof To show Part 1, let / G BZ'^, k £ Z'^ where B is a non-invertible matrix. Let f £ L (M ) . 
We have 

nUiT^Hl-^f = e-^-^'''^)/ = XI {k) f. 
If G GL {d, Z) then the commutator subgroup of G is trivial, and G is an abelian group isomorphic 
to Z'^xBZ'^. For Part 2, let us suppose that B e GL (d, Q)-GL {d, Z) .So,B = [pij /qij]^<ij<d ^^ere 
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Pij,qij are integral values, gcd (pij,qij) = 1 and qij ^ for 1 < i, j < d. Let m = Icm (qij) -^^^ . j^^. 
Clearly 

(k) = 1 for all I e BZ"^, and k £ 1'^. 

Thus, [G, G] is a finite abelian proper closed subgroup of the circle group. As a result [G, G] 
is cyclic. For Part 3, if i? G GL((i, M) — GL{d,Q) then the commutator subgroup of G is not 
isomorphic to a finite subgroup of the torus. That is, there exist / G BZ'^, /c G Z'^ such that the set 
{x™ (k) ■ m £ Z} is not closed and is dense in the torus T. □ 

Example 3.2. Let G be group generated Tk,Mi such that k G Z^,Z G BZ'^ 



B 

then, [G,G] is isomorphic to Zgo- 



1/2 1/5 
2/3 -3/4 



Example 3.3. Let G be group generated T^^Mi such that k G Z^,/ G B7? such that 

B : 



\/2 1 
-1 2 



i/ien [G, G] is isomorphic to an infinite subgroup of the circle. 

Assuming that B is in GL (d, Q) , we will construct an abelian normal subgroup of G. For that 
purpose, we will need to define the groups 

A^i = (Tk, Mil k£ B-^^'Z'^, I G BZ"^ 

and 

N2 = (Tk,Mi \ k£ B~^'''Z'^ nZ'^,1 £ B2 



Notice that in general A'^i is not a subgroup of G because the lattice Z'^ is not invariant under the 
action of B~*^ if B~^^ has non integral rational entries. However, the group A^i will be important 
in constructing the unitary dual of G, and we will need to study some of its characteristics. 

Lemma 3.4. // B is an element of GL (d, Q) — GL {d, Z) then 

Ni = (Tk,Mi I k G B-^''Z'^,l G BZ"^] 
is an abelian group. 

Proof. Given k G B'^'^Z'^ and / G BZ"^, we recall that 

Since k G B'^'^Z'^, and I G BZ'^ then there exist k', I' G such that 

Thus, for any given k G B^^'^'Z'^, and / G BZ'^, T^MiT^^ Mj^^ is equal to the identity operator. It 
follows that the commutator subgroup of A'^i is trivial. □ 

We recall the following lemmas from [8j. 

Lemma 3.5. Given two lattices ri,r2, rinr2 is a lattice in if and only if there exists a lattice 
r that contains both Fi and F2. 
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Definition 3.6. Let T be a full-rank lattice in with generators 













vf 




vi 


•) ' ' ' 1 


.4. 



The matrix of order d below 



is called a basis for T. 

Lemma 3.7. Let ri,r2 be two distinct lattices with bases J,K respectively. Ti + Fi n r2 are 
two lattices in M.'^ if and only if JK~^is a rational matrix. Moreover 

dim (Fi + Fa) + dim (Fi n Fa) = dimFi + dimFi 

Remark 3.8. /f Fi n F2 is a lattice, there is a well-known technique in [8] (see page 809) used 
to compute the basis of the lattice Fi R F2. We describe the procedure here. Let J,K be bases for 
lattices Fi, and F2 respectively. First we compute the d x 2d matrix [J \K] . Secondly, we evaluate 
the Hermite lower triangular form of [J\K] . This form has the structure [L\0] , and is obtained 
as [J \K] E = [L\0] , where E is matrix of order 2d obtained by applying elementary row operations 
to [J \K]. In fact, E is a block matrix of the type 



E 



R S 
C D 



and R, S, C, and D are matrices of order d. Finally, a basis for the lattice Fi n F2 is then given by 
KD. That is Fi n F2 = {KD) T^. 



— GL (d, Z) then B n Z*^ is a full-rank lattice subgroup 
Proof. The fact that B~^^'L'^ n Z'^ is a fuh-rank lattice follows directly from Lemma l3.7i □ 



Corollary 3.9. // B is in GL {d. 

ofW^. 



We assume that B is an element of GL (d, Q) — GL {d, Z) . We will prove that A'^2 is a normal 
subgroup of G. However it is not a maximal normal subgroup of G since it does not contain the 
center of the group. Thus, A'^2 needs to be extended. For that purpose, we define the group 

N = (Tk,Mi,T I k G B~*''Z'^nZ'^,l £ BZ'^,t e [G,G]^ < G. 

Proposition 3.10. If B is in GL{d,Q) then N is an abelian normal subgroup of G. 

Proof. From Lemma [3.41 we have already seen that T^ commutes with Mi for arbitrary k G i?^*''Z'^n 
U^, I £ BZ'^. Since [G, G] commutes with and Mi for k £ B'^'^Z'^nZ'^, I £ BZ'^, then N is abelian. 



For the second part of the proof, let k £ B 



-tr'-. 



,l£ BZ and s £ Z . First, we compute the 



conjugation action of the translation operator on an arbitrary element of A^. Let s £ Z"^, 

Ts {TkMiT) T-^ = Te-2-<''^)r,,Mz. 
Next, we compute the conjugation action of the modulation operator on an arbitrary element of 

Ms{TkMiT)M-^ = TTkMi. 



Clearly, GNG ^ C N. Thus, is a normal subgroup of G. 



□ 
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Lemma 3.11. Assuming that B is in GL (d, Q) — GL (d, Z) , the quotient group 



is isomorphic to a finite ahelian group. 

Proof. Since B~^^'Z,'^ n Z'^ is a full-rank lattice, there exists a non-singular matrix A such that 
B'^^'Z"^ nZ'^ = AZ'^. Thus, referring to the discussion ^ (page 95), the index of B'^^'Z'^ n Z'^ in Z'^ 
is {Z'^ : B-^'^Z'^ n Z'^) = \detA\ . As a result. 



B-^^'Z'^ n z<^ 

is a finite abelian group. □ 

Lemma 3.12. Assuming that B is in GL{d,Q) — GL {d,Z) , the group G/N is a finite group 
isomorphic to 



B-^^'Z'^ n Z'^ 

Proof. First, there exit ki,k2 ■ ■ ■ , G Z'^ such that 

= { i""''^^' ^ ' i""''^^' n z^) , . . . , fc„ [b-^'z'^ n z^) } 

Also, 

G/N = {N,n,N,--- ,n^N} 



and this completes the proof. □ 

4. The Plancherel Measure and Application: The Integer Case 

If i? G GL {d, Z) then G is commutative and isomorphic to Z'^ x BZ'^. In this case, the unitary 
dual, and the Plancherel measure of 

G = (Tk,Mi ■.k£Z'^,l£ BZ'^ 



are well-understood through the Pontrjagin duality. Let L be the left regular representation of G. 
The unitary dual of G is isomorphic to 



Z'^ B-t^'Z'^ 

and the Plancherel measure is up to multiplication by a constant equal to 

dxdy 



\det(B' 



-tr\ 



which is supported on a measurable set A C M^'^ parametrizing the group ^ x More 
precisely, the collection of sets 

|a + j : j € Z^ X B-^'Z'^^ 

forms a measurable partition for M^'^ and A is called the spectrum of the left regular representation 
of TT. It is worth noticing that there is no canonical way to choose A. Moreover, via the Plancherel 
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transform, the left regular representation of x BT!^ is decomposed into a direct integral decom- 
position of characters ^® X{x,y) |det i3| dxdy acting in f® C |det B\ dxdy = (A) . 

Now, we will discuss some application of the Plancherel theory of G to Gabor theory. Let 
B eGL {d, Z) , and let G = Z'^ x BZ'^. There is a representation of G where vr : G ^ [/ (L^ (R^)) , 
such that G is the image of G via the representation vr. The representation vr is called a Gabor 
representation of G. Since G is abelian, once a choice for a transversal of ^ x ^J^^^^ is made, 
there is a decomposition of vr into irreducible representations of the group G 

Xf lc"WC^/^(?) (6) 

acting in Xf C (g) C"('^)(i// (?) • The function n : A N U {0} is the multiplicity funct ion of the 
irreducible representations appearing in the decomposition of vr. 

Let us define Wf : (W^) ^ P ( g) where 



Wfh{kJ) = {h,TkMif). 
We recall that vr is admissible iff Wf defines an isometry on 
Lemma 4.1. Let B G GL (d, Z) . it is admissible if and only if 

\{h,nMif)\' 

That is f is a Gabor Parseval frame. 
We recall that the Zak transform is a unitary operator 

Z : (m-^) ^ ([0, l)"* X [0, l)'^) ^ / Cdxdy 



[0,1)'' X [0,1)" 

where 

Zf {x, y) = Y f {x + m) exp {2m (m, y)) . 

It is easy to see that if € GL (d, Z) 

Z (TkMif) (x, y) = exp (-27r« {k, y)) exp (27ri (/, x)) Z/ (x, y) . 
Thus, the Zak transform intertwines the Gabor representation with the representation 



[0,l)''x[0,l)'' 



That is, there is no Parseval frame of the type vr yOj f. Moreover, the Gabor representation is 
unitarily equivalent to the direct integral 

I 

Xf lc|dcti3|d^(?) . 

A 

Proof. First, let us notice that if S G GL{d,Z) then |det-B^*''| < 1. As a result, there is a 
measurable set A C M^*^ tiling R^'' by the lattice Z'^ x B~^^Z'^ such that A is strictly contained 
in [0, l)*^ X [0, l)"^ (see [E]). Thus if |detB| 7^ 1, the representation /jQ-^^d f^]^^dX(x,y)dxdy cannot 



X{x,y)dxdy. (7) 
Now, we would like to compare the representations given in ([6]) and ([7]). 

Proposition 4.2. If B G GL (d, Z) and |det B\ ^ 1 then the Gabor representation is not admissible. 
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d , . T7I ^ Tn,2d l_ m ^ rn i\d 



be contained in /® X{x,y) |det (S*'') | dxdy. Picking A = [0, ifxEc M^d g^^.^^ ^^lat E C [0, l)"* , we 
obtain 



(8) 



We now claim that the multiphcity function is given by 

n (0 = # ({i G S-*'-Z'^ : + j} n [0, \f 7^ 0}) = |det S| . 

To show that the above holds, we partition [0, 1)'^ x [0, l)*^ into |deti?| many subsets A*^ such that 
each set A*^ is a fundamental domain for Z*^ x 5~*^Z^. Writing 

|detB| 

[o,i)<^x[o,i)'^= u A^ 



we obtain 



fe=i 



|det B\ 



TT 



- / . Xix,y)dxdy^ / X(x,y)dxdy 
7[0,l)<^x[0,l)'' jfcti -^Afc 

- © / X{x,y)dxdy ^ X{x,y)dxdy 
k=i •'^ •'^ k=i 

rS) 

- / X? ® lcldeti3|d//(<j) . 



□ 



Example 4.3. Let 

G={Tk,Mi\keZ,le 3Z). 
T/ie spectrum of the left regular representation of G is given by 

A = [0, 1) X [0, 1/3) 

and TT = (8) Ic^dfj, (ij). Thus, tt is not an admissible representation since tt = L © L © L. 

Example 4.4. Xei 

G={Tk,Mi I fee Z^Z G SZ^) 
' 3 1 



B 



1 2 



r/te spectrum of the left regular representation is 



[0,lf X 



5 5 



[0, if . 



In the graph below we illustrate the idea that there exists a collection of sets 

such that each set is a fundamental domain of [0, 1) x B~^^ [0, 1) and the spectrum of tt is 
given as follows 

5 

[0,l)^x[0,l)^=|jA^ 

k=l 
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Thus, tt = L(BL(BL(BL(BL. As a result the Gabor representation ir is not admissible. 




A projection of the spectrum of tt in M?. 

(fC is a shift-modulation invariant 



Remark 4.5. Let IC be a n-invariant closed subspace of L'^ 
space [2jj. There exists a unitary operator 

rS) _ 
intertwining the representations vrj/C with 

andn{q) < |deti?| a.e. As a result, we have more or less a characterization of shift-modulation 
invariant spaces in the specific case where B has integral entries. 

Proof. The proof follows from the fact that (7r|/C,/C) is a subrepresentation of the Gabor represen- 
tation of G. □ 

Definition 4.6. Two unitary representations {tti,Hi) , {tt2,H2) of a group X are quasi- equivalent 
if there exit unitarily- equivalent representations pi,P2 such that is a multiple of 7:^ for k = 1,2. 

Remark 4.7. If B £ GL[d,'L) then the Gabor representation vr is quasi- equivalent to the left 
regular representation of G. 

□ 



Proof. The proof is a direct application of Proposition 14.21 

Remark 4.8. In the case where B is the identity matrix, vr is an admissible representation. In 
fact, a well-known admissible vector is the indicator function of the cube [0,1)'^. 

5. The Plancherel Measure and Application: The Rational Case 

In this section, we assume that the given matrix B has at least one rational non-integral entry. 
We recall from Lemma |3. II that 



G 



Tk,Mi ■.kez'^,le BZ'^ 
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is not commutative but is a discrete type I group. Thus, its unitary dual exists, and its Plancherel 
measure are computable. Using Mackey's Machine and results developed by Ronald Lispman 
and Adam Kleppner in [12], we will describe the unitary dual of the group G, and a formula for 
the Plancherel measure. We recall that if B is in GL (d, Q) — GL {d, Z) then G contains a normal 
subgroup 

N = (Tk,Mi,T I k G B-^'"L'^r\'L'^,l G BZ'^,t £ [G,G] 



which is isomorphic with a direct product of abelian groups. Since is isomorphic to 



X Ba 



its unitary dual is a group of characters. The underlining set for the group G is Z' 
and we define the representation tt of the group G as follows. 

Lemma 5.1. Let B G GL{d,Q) . If tt is admissible then exists a function f £ L'^ 
given h £ L"^ {R'^) , 

'h,nMd#{[G,G]y^''f' 



X BZ'^ X Zm, 

(9) 

R"^) such that 



E 

TkMieG 



That is vr (G) # ([G, G])^/^ / is Parseval Gabor frame. 

Proof Let /i G (R'^) . If vr is admissible, and if / is an admissible vector, 



h,TkMie^^''^f 



^ ^ \{h,T,MJ)\' 
Y,*i[G,G])\{h,nMif)\^ 



TkMi 



J2 \{h,TkM,#{[G,G])'^'f 



□ 

Using the procedure provided in Remark 13.81 we construct an invertible matrix A with integral 
entries such that B~^^Z'^r\Z'^ = AZ'^. Thus, the unitary dual of A^ is isomorphic to the commutative 
group 

Rd. l^d 

X Zm- 



A-t^'Z'i B-^-'Z'^ 

Letting Ai C M'^ be a measurable fundamental domain for A~^^Z'^ and A2 C M'^ a measurable 
fundamental domain for the unitary dual of A^ is parameterized by the set 

I (71, 72,0-) : 71 G Ai,72 G A2,fT G Z„| . 

Next, the action of a fixed character of A^ is computed as follows. 



2-!Tie 



exp [27ri (71, k)] exp [27ri (72, /)] exp [27rzcT0] 



X(7i,72,<t) \ TkMie 

We recall one important result due to Mackey which is also presented in p!3]. We will need this 
proposition to compute the unitary dual of the group G. 
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Proposition 5.2. Let N he a normal subgroup of G. Assume that N is type I and is regularly 
embedded. Let it be an arbitrary element of the unitary dual of N. G acts on the unitary dual of N 
as follows: 

X ■ TT (y) = TT (x^^yx) ,x G G,y G N. 
Let be the stabilizer group of the G-action on vr. 

G = |lndQ^ (v) : v G Gvr and v\N is a multiple o/7r| . 

■K&N/G 

Now, we will apply Proposition 15.21 to 

G = (Tk,Mi -.kelf^le B7'^^ 



First, we recall from Lemma 13.121 that G/N is a finite group, and thus G is a compact extension 
of an abelian group. Referring to [12] I Chapter 4, G is type / and is regularly embedded. Let 
P £G,k£ B-^'^Z'^ r\Z'^,l£ BZ'^ and let X(7i,72,a) be a character of N. We define the action of G 
on the unitary dual of N multiplicatively such that for P £ G, 

P ■ X(71,72,.) {TkMie^^^') = Xi-y,,-y,,a) {P'' (TfcM.e^-^) P) . (10) 

Definition 5.3. We define a measurable map p : M'^ — )■ A2 such that p (x) = y^ if and only is 
the unique element in A2 such that x = yx + 1, where I G B~'^^Z'^. 

Since the collection of sets 

[k^+j-.j^LB-'^Z"] 

is a measurable partition of M'^ then the map p makes sense. 
Lemma 5.4. For seZ'^,le BZ'^ and e^'^'^ G [G, G] we have 

(a) Tg ■ X(7i,72,(t) ~ X(7i,p(72+CTs),o-) 

(b) Ml ■ 

X{7i:72,o") ^(71172,0") 

(c) e • X{7i,72,o-) — X(7i,72,o') 

Proof. Since e^'^'^ is a central element of G then clearly Part 3 is correct. Now, for part 1, let 
Tfc, M/, such that k £ B'^^'Z'^ nZ'^,1 £ BZ'^ and s £ Z"^ 



x(7,,72,.)((r-irfcM;r,e2-^)) 

X(7i,72,.) (T.M,e2-(^+<''^)) 



exp [27rz (71, k)] exp [2-Ki (72, /)] exp [Inia {9 + (/, s))] 

exp [27rz (71, k)] exp [27ri (72, /)] exp pyrifj (6*)] exp pvrifT (/, s) 

exp [27rz (71, k)] exp [27ri (72 + sa, I)] exp pTrio" (6*)] 



Thus, T, ■ X{7i,72,a) = X(7i,p(72+<xs),a)- For part 2, since k E B'^^'Z'^ nZ^Je BZ'^ 

X(7„7„.) [m_i {TkMie^^'') Ml) = X(7i,72,a) (M.^M^TfcM^e^-^e^-^^'''^^) 



X(7,,72,.) ( TfcMze^-^ 
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□ 

Lemma 5.5. The stabilizer group of a fixed character X(7i,72,(t) unitary dual of N is given 

by 

TkMie^""'^ £G \ ak = B^^'^j for some j £ "^"^ 
/ G BZ'^, e2^^^ G [G, G] 



„ / Ajfiviic t i_r I Dfij — u J joi some j t ^ \ /i i \ 

^(71.72,0-) = \ 1 ^ U^d „27ri6» ^ \n ni / \^^) 



Thanks to (fTTj) . we may write 

G(^,,^2,^) = A [a] Z'^ X BZ"^ x [G, G] 

where A (a) is a fuh-rank matrix of order d and A (a) Z'^ > AZ'^. For a fixed element (71, 72, cr) in 
the unitary dual of N, we define the set 

l(^ = A {a) X A2 X [gTg] (12) 

where A (a) is a fundamental domain for fltrr^^ such that Ai is contained in A (a) . Since we 
need Proposition 15.21 to compute the unitary dual of G, we would like to be able to assert that 
if X(7i.72,o-) is a character of the group A^, it is possible to extend X(7i,72,ct) *o ^ character of the 
stabilizer group G(^^ .^j.o-)- However, we will need a few lemmas first. 

Lemma 5.6. Let A = (r/, 72, cr) E U^j such that ?7 = 71 + A~^^j for some j G Z"^. If X(7i+yl-*''j,72,cr) 
is a character 0/ G(^^ ,y2,o-)i then 

X(7i+A-"-j,72,f7) (TfcM^e^''*^^ = X(7i,72,(t) (TfoM^e^"*^^ 
Proof Let TkMie^'^'^ G A^. Since A; G AZ^, there exits k' G Z'^ such that k = Ak' 

XX (TfcMze^-^) = exp (liri (71, k)) exp (27ri (^"^'^j, A:» exp (27ri (72, /)) e'™' 
= exp (27ri (71, A;)) exp {ini (j, k')) exp (27ri (72, 1)) e^™^ 
= exp {2TTi (71, A:)) exp (27ri (72, /)) e^™^ 



X(7i,72,fT) [TkMie 



Lemma 5.7. For a /ixeci 

(71,72,0-) G Ai X A2 X 
in the unitary dual of N, if X = [rj, 72, f) G Ua, = 71 + A^^'^j for some j G Z'^ then 

X\ [^(71,72,0"), ^(71,72,0")] ^■ 

Proof. Let T^Mit G G(^^ ,^2,0-)- I* suffices to check that 

XA {TkMiTT^^M^\-^) = 1 

where r G [G, G] . First, we observe that 

XA (TkMirT-H-I-^T'^) = XX {nMiT-'M-') = exp [-2^i {ak, I)] . 

Applying Lemma [53} since G G^^^ ,^2,0-) there exists some p £ Z'^ such that 

XA {TkMirT^^M^^T-') = exp [27ria />] . 

Secondly, using the fact that / G BZ'^ there exits /' G Z'^ such that 

XA inMiTT-Hl-^T-^) = exp [27ria {B'^^'p, Bl')] = 1. 



□ 



□ 
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The following lemma allows us to establish the extension of characters from the normal subgroup 
N to the stabilizer groups. 

Lemma 5.8. Fix (71, 72,17) in the unitary dual of N. Let A = (r/,72,0") € U^- Then x\ defines a 
character on G(^^ .^j^o-)- 

Proof. Given Tk^Mi^e^'^^^^ and T^.^ M/^ e^'^*^^ g G(^^_^2^<^), it is easy to see that 



where e^'^*^'^'^^} ^ [^(71,72,(7)) ^2,0-)] • We want to show that x\ defines a homomorphism from 
^(7i,72,<t) into the circle group. Since xa [G'(7i,72,a)> ^'(^^^^^.(t)] = 1 then 

= XA (r.,+.,M,,H.,,e2-(''^+^^+('-^^))) 

= exp {2m {r,, h + A:2» exp {2-Ki (72, + /2)) e2™(^i+^2)g2.*(a(«i,fc2» 
From Lemma E31 e^'^^^'^^'i'^^}) = 1 ^nd 



= exp {2TTi {rj, ki + k2)) exp (27ri (72, h + ^2)) e2-*'^(^i+^^) 
Now, using Lemma 15.61 

XA ((Tfc^M^^e^-^i) (rfc,Mz,e2-^^)) = xa [n^Mi^e^^''^) xx [n,Mi,e^^''^ 
Thus Xa defines a character on ^(^^ ,^2.0-)- 1^ 

Remark 5.9. Fix (71, 72, c) ^'t^ ^/^e unitary dual of N. Let r/ = 71 + j € Z^o- where j S Z^. T/ie 
character X(?7,72,cr) ^-^ called an extension 0/ X(7i,72,(j) /toto to f/ie stabilizer group G^y-^^^^,^)- 

Proposition 5.10. The unitary dual of G is parameterized by the set 

S = y where = |J (Ai x x {a}) , 

and E(j is a measurable subset of A2 satisfying the condition 

[J p{E„ + as) = A2. 

Proof. Fixing a £ [G, G], recall that 

G ■ (71, 72, c^) = [{li,P (72 +(Ts),a):s£ Z^} . 

For a fixed ci G [G, G] , we pick a measurable set Eo- C A2 such that 

U p(E, + cTs) = A2. (13) 

The set 

n= \J (Ai X E, X {a}) 
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parameterizes the orbit space N/G. Thus, fohowing Mackey's result (see Proposition 15. 2p . the 
unitary dual of G is parametrized by the set 

(71 .72,0") GO 

□ 

Following the description given in [12], Section 4, let X(7i,72,<7) ^ ^ and let x\^^^^^^„) = X(7i+A-t'-j,72,<7) 
be its extended representation from to ,^2 0-). Let C be an irreducible representation of 

^(71,72,0-)/-^ and define its lift to G{'^^,'y2,a) which we denote by Q. For a fixed (71,72,0",^) we 
define the representation 



'^(7i:72,o-,C) 



^(71,72,^,0 = I"dG(,^,,2,.) W7i,72,<.) ® ^] 

acting in the Hilbert space 

u (TuP) = [(4,,2,<.) ® C) (^')] u (T,) , i (14) 
where P € G(^j^^2,o-) J 
which is naturally identified with 

The inner product on ^{71,72, o-.C) given by 

Notice that the number of elements in G/G(^^^^2,o-) is bounded above by the order of the finite 



group 



which has precisely |det A| elements. 

Remark 5.11. Every irreducible representation of G is monomial. That is, every irreducible 
representation of G is induced by a one- dimensional representation of some subgroup of G. Given 
u E "^(71,72,0-,^)' 



P{7i,72,-,C) [TkMie'^'') u (T,) = u ^T^M^e^^^^j 
(i u ^(TfcM/e^'^*^) ""^ is computed by following the rules defined in [T4\ ) where 



and 



zs a sei 0/ representative elements of the quotient group -^z- 

The lemma above is a standard computation of an induced representation. The interested reader 
is referred to f5] 

Next, we define the set 

= I 1^^^(7.72..) (^(7.72,.) : C e G^,Z^^/N 1 _ 
1 (71,72,0-) G Ai X X {cr} J 
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A more convenient description of the unitary dual of G which will be helpful when we compute the 
Plancherel measure is 

S= J S,. (15) 

Now that we have a complete description of the unitary dual of the group G, we will provide a 
computation of its Plancherel measure. 

Theorem 5.12. The Plancherel measure is a fiber measure which is given by 

dnii (71) dm2 (72) dm^ (a) dm^ {Q) 



dfJ' (P(7i,72,<T,C)) 



|det^r^m2 (E^) 



The measures dmi, dm2 are the canonical Lebesgue measures defined on Ai and A2 respectively. 
The measure dm^ is the counting measure on the unitary dual of the commutator [G, G] and dru/^ 

is the counting measure on the dual of the little group ,o-) with weight (^(^^ -^2,0-) : ■ 

Moreover if L is the left regular representation of G, the direct integral decompositon of L into 
irreducible representations of G is 

/'(7i>72,o-,C) ® lc"(Ti'T2'<''f) dl^ (P(7i ,72,^,0) 
acting in 

and 

n(7i,72,o-,C) = card (G/G(^^_^2_^)) . 

Proof. The theorem above is an application of the abstract case given in [12] II (Theorem 2.3), and 
the precise weight of the Plancherel measure is obtained by some normalization. □ 

Let us suppose that B is in GL {d, Q) — GL (d, Z) . Let if be any type I representation of G. Then, 
there is a unique measure, Ji defined on the spectral set S such that (p is unitarily equivalent to 



/'(7l,72,fT,C) lci"(71,72.'^>C)C^ (^(71,72,(7,0) (^^) 

acting in 

.n(7i,72,<x,C) ^ {Pi^„^,,a,0) 



where m is the multiplicity function of the irreducible representations occurring in the decomposi- 
tion of (p. 

Proposition 5.13. The representation ip is admissible if and only if 

(a) c?/" (/O (71,72, (T,c)) absolutely continuous with respect to the Plancherel measure ofG. 

(b) m(7i,72,cr,C) < card (G/G(^^^^2_„)) . 

Proof. See [6] page 126 □ 

Remark 5.14. If B is in GL{d,Q) — GL{d,'Z) and |deti?| < 1 and the Gabor representation vr 
(0) is unitarily equivalent to M(?^) then vr is admissible and the Proposition above is applicable. 

Proof. This remark is just an application of the density condition given in Lemma 12.61 □ 



PLANCHEREL MEASURE AND ADMISSIBILITY 18 

6. Non-Type I Groups 

In general if B has at least one non-rational entry, then the commutator subgroup of G is a 
non-finite abelian group. In this section, we will consider the case where d = 1, and 

G={Tk,Mi\keZ,le aZ) 

where a is irrational positive number. Unfortunately, the Mackey machine will not be applicable 
here, and we will have to rely on different techniques to obtain a decomposition of the left regular 
representation in this case. Let EI be the three-dimensional connected, simply connected Heisenberg 
group. We define 

1 m I 

\,m,k= 1 k ■.{m,k,l)eZ^ 
1 

Lemma 6.1. There is a faithful representation 0" ofV such that 

e° (Pi,o,o) = Xa (0 , (Po,™,o) = Tm, and G° (Po,0,fc) = Mka- 

Proof. 0" is the restriction of an irreducible infinite-dimensional representation of the Heisenberg 
group ^ to the lattice T. Since 

'10 



kerG'^ 



1 
1 



the representation is clearly faithful. 

Thus, G = T via G" and for our purpose, it is more convenient to work with F. We define 

^l = {Po,„^,fc:KA:)GZ2}. 

Let Lin be the left regular representation of the simply connected, connected Heisenberg group 

1 X z 
1 y 



□ 



1 



: [z,y,x) G 



In fact, it is not too hard to show that F is a lattice subgroup of I 
transform of the Heisenberg group. We recall that 

V:L^(B.)-^ (R)(g> (R)\X\dX 

where the Fourier transform is defined on L^(EI) Pi L^(EI) by 



Let V be the Plancherel 



rif)i\) 



tta in) f (n) dn 



where 

TTA (n) / (t) = TTX (P.,.,,) / (t) = e2--Ag-2-Aj;t^ _ ^) ^ 

and the Plancherel transform is the extension of the Fourier transform to 
equality 



inducing the 



2 

L2(H) 



\\Vif)iX)\\l,.\X\dX. 



In fact, II • 1 1 -^5 denotes the Hilbert Schmidt norm on 
rank-one operators in L'^ (M) (8) (R) . We have 



{u(^ v,w y) 



US 



{u,w) 



{v,y)LHM 



. Let n (g) t> and w y he 
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It is well-known that 



Ln=VoLaoV 



VTA ® 1l2(k) |A| d\, 



where 1^2 (r) is the identity operator on (M) , and for a.e. A G M*, 

P(Lh(x)0)(A) = ^a(x)oP0(A). 

Let 

{ua:Ag[-1,0)U(0,1] and ||uA||i2(K) = l} 

be a measurable field of unit vectors in (M) . We define two left-invariant multiplicity- free 
subspaces of (H) such that 



H 



H 



V 



-1 



(0,1] 



{M)®ux\\\d\ , 



V 



-1 



-1,0) 



(g)UA |A| d\ 



and H + , H are mutually orthogonal. The following lemma has also been proved in more general 
terms in [11]. However the proof is short enough to be presented again in this section. 

Lemma 6.2. The representation (LulTiH is cyclic and H + admits a Parseval frame of the 
type Lji{T)f with 11/11^2(15) 

Proof. Let ^^(EI). 



1 

2- 



L2(H) 

7er 



E 

7er 



(0,1] 



EE 

7ierijgz 



(0,1] 



Since je^'^*-''^ : j G Z} forms a Parseval frame for ((0, 1]) , we have 



Ek'^'^nw/) 



L2(H) 



7er 



LetP(/)(A)|A| 



1/2 



7i6ri 

1 1/2 



(0,1] 



P0(A),^a(7i)^(/)(A)|A|1/2) |A|dA. 

I rib 

5 ua a.e. Then 



|A|'/ WA(g)UA G L 
2^ (0,Le (7)/>L2(e) 



7Gr 



(0,1] 



5 (A), VTA (7i)(|A|^/=^WA 



|A|dA 



(17) 



where VcpiX) = sa (Xi ua. Notice that by definition vta (71) f = exp {2-Ki\jt) f (t — k) where (Aj, k) £ 
XL X Z with A G (0, 1] . By the density condition given in Lemma |2.6^ it is possible to find va such 
that the system 

{tta (71) (va) : 71 e Ti} 

is a Parseval frame in 1? (M) for a.e. A G (0, 1] . So let us suppose that we pick / G H + so that 

P(/)(A) = |Ari/VA®UA. 
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L2(H) 

7er 



/ yZ (sA,7rA(7l)(vA)>L2( 

\\vm\\ls\Mdx 



\X\dX 



To prove that 



L2(H) 



(0,1] 
2 

L2(e) • 

^ , we appeal to Lemma 12.71 and we obtain 
|A|-V2- 



va 



2 



-1 II l|2 

I|va|Il2( 



= |Ar^vol(AZ X Z) 
= 1. 

The above holds for almost every A G (0, 1]. Next, since ||'P/(A)||^_5 = 1 we obtain 



2 

L2{H) 



Similarly, we also have the following lemma 



\X\dX = -. 



□ 



Lemma 6.3. The representation (Le|r,H ) is cyclic. Moreover H admits a Parseval frame of 
the type Le (L) h and 

Il2{H) = 2' 



Lemma 6.4. Let H = 
H. 



> H . Then there exists an orthonormal basis of the type Le (F) / for 



We remark that in general the direct sum of two parseval frames in H, and K is not an even a 
Parseval frame for the space H (B K , unless the ranges of the coefficients operators are orthogonal 
to each other. A very clever proof of Lemma 16.41 is given by Currey and Mayeli in [3], where they 
show how to put together / and h in order to obtain an orthonormal basis for H. Now, we are in 
a good position to obtain a decomposition of the left regular representation of the time-frequency 
group. First, let us define 

' r 1 / 1 
K=< 1k ■.{k,l)£l? 
|_ 1 

It is easy to see that K is an abelian subgroup of P. Let L be the left regular representation of P. 
Lemma 6.5. A direct integral decomposition of L is obtained as follows 

7rx\X\dX^I I Ind^ (;t(|;^|,))|A|dMA (18) 



-1,1] 



1,1] J[o,\x\) 



acting in the Hilbert space 



1,1] ■^[0,1A1) 



\X\dtdX 
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Proof. First, let us define Rj : H ^ P (F) such that Rjg (7) = {g, L (7) /) . Using Lemma [6^ (see 
[3] also), we construct a vector / G H such that Rf is an unitary. As a result, the operator RjoV"^ 
is unitary and 



o V-') o 1^^® VTA |A| dX^ (•) o (Poi?-i) = L (•) 



(19) 



where 



Thus, 



VTA 



1 m I 
1 
1 



/ (t) = e2rfAg2^ifcAi^ _ 



VTA |A|(iA^L. 



-1,1] 

Notice that (ttaIF, (R)) is not an irreducible representation. However, we may use Baggett's 
decomposition given in [T]. For each A G [—1,1] , the representation tta is decomposed into its 
irreducible components as follows: 



VTA 



[0,|A|) 



Ind^ (x(|A|,i)) dt 



(20) 



where 



X{\x\,t) 



1 / 
1 k 
1 



exp(27ri(|A|/ + t/c)). 



Combining ()19p with (I20p . we obtain the decomposition given in (llSh . 



□ 
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